{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Vertically Federated XGB (SecureBoost) Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">The following codes are demos only. It's **NOT for production** due to system security concerns, please **DO NOT** use it directly in production."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparison with XGB\n",
    "\n",
    "In this notebook, we are going to compare the performance of our implementation of secureboost vs XGB in cleartext.\n",
    "\n",
    "In the end we will give a comparison between the two models on the same datasets with respect to AUC.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/zoupeicheng.zpc/miniconda3/envs/py310/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "2024-04-22 15:36:44,857\tINFO util.py:154 -- Missing packages: ['ipywidgets']. Run `pip install -U ipywidgets`, then restart the notebook server for rich notebook output.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The version of SecretFlow: 1.5.0b0\n"
     ]
    }
   ],
   "source": [
    "import pprint\n",
    "import time\n",
    "from typing import List\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import secretflow as sf\n",
    "import spu\n",
    "import xgboost as xgb\n",
    "from secretflow.data import FedNdarray, partition, PartitionWay\n",
    "from secretflow.data.split import train_test_split as train_test_split_fed\n",
    "from secretflow.data.vertical import VDataFrame\n",
    "from secretflow.device.driver import reveal, wait\n",
    "from secretflow.ml.boost.sgb_v import (\n",
    "    get_classic_lightGBM_params,\n",
    "    get_classic_XGB_params,\n",
    "    Sgb,\n",
    ")\n",
    "\n",
    "from secretflow.ml.boost.sgb_v.core.params import xgb_params_converter\n",
    "from secretflow.ml.boost.sgb_v.model import load_model\n",
    "from secretflow.preprocessing import LabelEncoder\n",
    "from secretflow.utils.simulation.datasets import (\n",
    "    load_bank_marketing,\n",
    "    load_bank_marketing_unpartitioned,\n",
    "    load_creditcard,\n",
    "    load_creditcard_unpartitioned,\n",
    ")\n",
    "from sklearn.metrics import roc_auc_score\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import LabelEncoder as SkLabelEncoder\n",
    "\n",
    "pp = pprint.PrettyPrinter(depth=4)\n",
    "\n",
    "# Check the version of your SecretFlow\n",
    "print('The version of SecretFlow: {}'.format(sf.__version__))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/zoupeicheng.zpc/miniconda3/envs/py310/lib/python3.10/subprocess.py:1796: RuntimeWarning: os.fork() was called. os.fork() is incompatible with multithreaded code, and JAX is multithreaded, so this will likely lead to a deadlock.\n",
      "  self.pid = _posixsubprocess.fork_exec(\n",
      "2024-04-22 15:36:47,049\tINFO worker.py:1724 -- Started a local Ray instance.\n"
     ]
    }
   ],
   "source": [
    "alice_ip = '127.0.0.1'\n",
    "bob_ip = '127.0.0.1'\n",
    "ip_party_map = {bob_ip: 'bob', alice_ip: 'alice'}\n",
    "\n",
    "_system_config = {'lineage_pinning_enabled': False}\n",
    "sf.shutdown()\n",
    "# init cluster\n",
    "sf.init(\n",
    "    ['alice', 'bob'],\n",
    "    address='local',\n",
    "    _system_config=_system_config,\n",
    "    object_store_memory=50 * 1024 * 1024 * 1024,\n",
    ")\n",
    "\n",
    "# SPU settings\n",
    "cluster_def = {\n",
    "    'nodes': [\n",
    "        {'party': 'alice', 'id': 'local:0', 'address': alice_ip + ':12945'},\n",
    "        {'party': 'bob', 'id': 'local:1', 'address': bob_ip + ':12946'},\n",
    "        # {'party': 'carol', 'id': 'local:2', 'address': '127.0.0.1:12347'},\n",
    "    ],\n",
    "    'runtime_config': {\n",
    "        # SEMI2K support 2/3 PC, ABY3 only support 3PC, CHEETAH only support 2PC.\n",
    "        # pls pay attention to size of nodes above. nodes size need match to PC setting.\n",
    "        'protocol': spu.spu_pb2.SEMI2K,\n",
    "        'field': spu.spu_pb2.FM128,\n",
    "    },\n",
    "}\n",
    "\n",
    "# HEU settings\n",
    "heu_config = {\n",
    "    'sk_keeper': {'party': 'alice'},\n",
    "    'evaluators': [{'party': 'bob'}],\n",
    "    'mode': 'PHEU',\n",
    "    'he_parameters': {\n",
    "        # ou is a fast encryption schema that is as secure as paillier.\n",
    "        'schema': 'ou',\n",
    "        'key_pair': {\n",
    "            'generate': {\n",
    "                # bit size should be 2048 to provide sufficient security.\n",
    "                'bit_size': 2048,\n",
    "            },\n",
    "        },\n",
    "    },\n",
    "    'encoding': {\n",
    "        'cleartext_type': 'DT_I32',\n",
    "        'encoder': \"IntegerEncoder\",\n",
    "        'encoder_args': {\"scale\": 1},\n",
    "    },\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "alice = sf.PYU('alice')\n",
    "bob = sf.PYU('bob')\n",
    "heu = sf.HEU(heu_config, cluster_def['runtime_config']['field'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:root:Create proxy actor <class 'secretflow.data.core.agent.PartitionAgent'> with party alice.\n",
      "INFO:root:Create proxy actor <class 'secretflow.data.core.agent.PartitionAgent'> with party bob.\n",
      "INFO:root:Create proxy actor <class 'secretflow.data.core.agent.PartitionAgent'> with party alice.\n"
     ]
    }
   ],
   "source": [
    "data = load_bank_marketing(parts={alice: (0, 16), bob: (16, 16)}, axis=1, full=True)\n",
    "label = load_bank_marketing(parts={alice: (16, 17)}, axis=1, full=True)\n",
    "\n",
    "bank_unpartitioned = load_bank_marketing_unpartitioned(full=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(45211, 17)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bank_unpartitioned.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bound method NDFrame.head of        age           job   marital  education default  balance housing loan  \\\n",
       "0       58    management   married   tertiary      no     2143     yes   no   \n",
       "1       44    technician    single  secondary      no       29     yes   no   \n",
       "2       33  entrepreneur   married  secondary      no        2     yes  yes   \n",
       "3       47   blue-collar   married    unknown      no     1506     yes   no   \n",
       "4       33       unknown    single    unknown      no        1      no   no   \n",
       "...    ...           ...       ...        ...     ...      ...     ...  ...   \n",
       "45206   51    technician   married   tertiary      no      825      no   no   \n",
       "45207   71       retired  divorced    primary      no     1729      no   no   \n",
       "45208   72       retired   married  secondary      no     5715      no   no   \n",
       "45209   57   blue-collar   married  secondary      no      668      no   no   \n",
       "45210   37  entrepreneur   married  secondary      no     2971      no   no   \n",
       "\n",
       "         contact  day month  duration  campaign  pdays  previous poutcome    y  \n",
       "0        unknown    5   may       261         1     -1         0  unknown   no  \n",
       "1        unknown    5   may       151         1     -1         0  unknown   no  \n",
       "2        unknown    5   may        76         1     -1         0  unknown   no  \n",
       "3        unknown    5   may        92         1     -1         0  unknown   no  \n",
       "4        unknown    5   may       198         1     -1         0  unknown   no  \n",
       "...          ...  ...   ...       ...       ...    ...       ...      ...  ...  \n",
       "45206   cellular   17   nov       977         3     -1         0  unknown  yes  \n",
       "45207   cellular   17   nov       456         2     -1         0  unknown  yes  \n",
       "45208   cellular   17   nov      1127         5    184         3  success  yes  \n",
       "45209  telephone   17   nov       508         4     -1         0  unknown   no  \n",
       "45210   cellular   17   nov       361         2    188        11    other   no  \n",
       "\n",
       "[45211 rows x 17 columns]>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bank_unpartitioned.head"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         no\n",
       "1         no\n",
       "2         no\n",
       "3         no\n",
       "4         no\n",
       "        ... \n",
       "45206    yes\n",
       "45207    yes\n",
       "45208    yes\n",
       "45209     no\n",
       "45210     no\n",
       "Name: y, Length: 45211, dtype: object"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bank_unpartitioned['y']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# preprocess the data\n",
    "encoder = LabelEncoder()\n",
    "data['job'] = encoder.fit_transform(data['job'])\n",
    "data['marital'] = encoder.fit_transform(data['marital'])\n",
    "data['education'] = encoder.fit_transform(data['education'])\n",
    "data['default'] = encoder.fit_transform(data['default'])\n",
    "data['housing'] = encoder.fit_transform(data['housing'])\n",
    "data['loan'] = encoder.fit_transform(data['loan'])\n",
    "data['contact'] = encoder.fit_transform(data['contact'])\n",
    "data['poutcome'] = encoder.fit_transform(data['poutcome'])\n",
    "data['month'] = encoder.fit_transform(data['month'])\n",
    "label = encoder.fit_transform(label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "encoder = SkLabelEncoder()\n",
    "for col in [\n",
    "    'job',\n",
    "    'marital',\n",
    "    'education',\n",
    "    'default',\n",
    "    'housing',\n",
    "    'loan',\n",
    "    'contact',\n",
    "    'month',\n",
    "    'poutcome',\n",
    "    'y',\n",
    "]:\n",
    "    bank_unpartitioned[col] = encoder.fit_transform(bank_unpartitioned[col])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bound method NDFrame.head of        age  job  marital  education  default  balance  housing  loan  contact  \\\n",
       "0       58    4        1          2        0     2143        1     0        2   \n",
       "1       44    9        2          1        0       29        1     0        2   \n",
       "2       33    2        1          1        0        2        1     1        2   \n",
       "3       47    1        1          3        0     1506        1     0        2   \n",
       "4       33   11        2          3        0        1        0     0        2   \n",
       "...    ...  ...      ...        ...      ...      ...      ...   ...      ...   \n",
       "45206   51    9        1          2        0      825        0     0        0   \n",
       "45207   71    5        0          0        0     1729        0     0        0   \n",
       "45208   72    5        1          1        0     5715        0     0        0   \n",
       "45209   57    1        1          1        0      668        0     0        1   \n",
       "45210   37    2        1          1        0     2971        0     0        0   \n",
       "\n",
       "       day  month  duration  campaign  pdays  previous  poutcome  y  \n",
       "0        5      8       261         1     -1         0         3  0  \n",
       "1        5      8       151         1     -1         0         3  0  \n",
       "2        5      8        76         1     -1         0         3  0  \n",
       "3        5      8        92         1     -1         0         3  0  \n",
       "4        5      8       198         1     -1         0         3  0  \n",
       "...    ...    ...       ...       ...    ...       ...       ... ..  \n",
       "45206   17      9       977         3     -1         0         3  1  \n",
       "45207   17      9       456         2     -1         0         3  1  \n",
       "45208   17      9      1127         5    184         3         2  1  \n",
       "45209   17      9       508         4     -1         0         3  0  \n",
       "45210   17      9       361         2    188        11         1  0  \n",
       "\n",
       "[45211 rows x 17 columns]>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bank_unpartitioned.head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Find best convergence round with XGB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0]\tvalidation_0-auc:0.87524\n",
      "[1]\tvalidation_0-auc:0.89267\n",
      "[2]\tvalidation_0-auc:0.90016\n",
      "[3]\tvalidation_0-auc:0.90033\n",
      "[4]\tvalidation_0-auc:0.90320\n",
      "[5]\tvalidation_0-auc:0.90478\n",
      "[6]\tvalidation_0-auc:0.90888\n",
      "[7]\tvalidation_0-auc:0.90987\n",
      "[8]\tvalidation_0-auc:0.91073\n",
      "[9]\tvalidation_0-auc:0.91157\n",
      "[10]\tvalidation_0-auc:0.91179\n",
      "[11]\tvalidation_0-auc:0.91261\n",
      "[12]\tvalidation_0-auc:0.91294\n",
      "[13]\tvalidation_0-auc:0.91373\n",
      "[14]\tvalidation_0-auc:0.91399\n",
      "[15]\tvalidation_0-auc:0.91512\n",
      "[16]\tvalidation_0-auc:0.91514\n",
      "[17]\tvalidation_0-auc:0.91579\n",
      "[18]\tvalidation_0-auc:0.91922\n",
      "[19]\tvalidation_0-auc:0.91998\n",
      "[20]\tvalidation_0-auc:0.92006\n",
      "[21]\tvalidation_0-auc:0.92013\n",
      "[22]\tvalidation_0-auc:0.92203\n",
      "[23]\tvalidation_0-auc:0.92222\n",
      "[24]\tvalidation_0-auc:0.92362\n",
      "[25]\tvalidation_0-auc:0.92499\n",
      "[26]\tvalidation_0-auc:0.92575\n",
      "[27]\tvalidation_0-auc:0.92590\n",
      "[28]\tvalidation_0-auc:0.92585\n",
      "[29]\tvalidation_0-auc:0.92599\n",
      "[30]\tvalidation_0-auc:0.92662\n",
      "[31]\tvalidation_0-auc:0.92652\n",
      "[32]\tvalidation_0-auc:0.92640\n",
      "[33]\tvalidation_0-auc:0.92616\n",
      "[34]\tvalidation_0-auc:0.92636\n",
      "[35]\tvalidation_0-auc:0.92609\n",
      "train set AUC score:  0.9427580112534844 test set AUC score:  0.9210143484657799 num_trees:  31\n"
     ]
    }
   ],
   "source": [
    "X = bank_unpartitioned.iloc[:, :-1]\n",
    "y = bank_unpartitioned.iloc[:, -1]\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.2, random_state=94\n",
    ")\n",
    "X_train, X_val, y_train, y_val = train_test_split(\n",
    "    X_train, y_train, test_size=0.2, random_state=94\n",
    ")\n",
    "\n",
    "# Use \"hist\" for constructing the trees, with early stopping enabled.\n",
    "clf = xgb.XGBClassifier(\n",
    "    tree_method=\"hist\",\n",
    "    n_estimators=100,\n",
    "    max_depth=5,\n",
    "    learning_rate=0.3,\n",
    "    max_bin=10,\n",
    "    early_stopping_rounds=5,\n",
    "    base_score=0.5,\n",
    "    eval_metric=\"auc\",\n",
    "    reg_lambda=0.1,\n",
    "    min_child_weight=0,\n",
    ")\n",
    "clf.fit(X_train, y_train, eval_set=[(X_val, y_val)])\n",
    "\n",
    "print(\n",
    "    \"train set AUC score: \",\n",
    "    roc_auc_score(y_train, clf.predict_proba(X_train)[:, 1]),\n",
    "    \"test set AUC score: \",\n",
    "    roc_auc_score(y_test, clf.predict_proba(X_test)[:, 1]),\n",
    "    \"num_trees: \",\n",
    "    clf.best_iteration + 1,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>XGBClassifier(base_score=0.5, booster=None, callbacks=None,\n",
       "              colsample_bylevel=None, colsample_bynode=None,\n",
       "              colsample_bytree=None, early_stopping_rounds=None,\n",
       "              enable_categorical=False, eval_metric=&#x27;auc&#x27;, feature_types=None,\n",
       "              gamma=None, gpu_id=None, grow_policy=None, importance_type=None,\n",
       "              interaction_constraints=None, learning_rate=0.3, max_bin=10,\n",
       "              max_cat_threshold=None, max_cat_to_onehot=None,\n",
       "              max_delta_step=None, max_depth=5, max_leaves=None,\n",
       "              min_child_weight=0, missing=nan, monotone_constraints=None,\n",
       "              n_estimators=31, n_jobs=None, num_parallel_tree=None,\n",
       "              predictor=None, random_state=None, ...)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">XGBClassifier</label><div class=\"sk-toggleable__content\"><pre>XGBClassifier(base_score=0.5, booster=None, callbacks=None,\n",
       "              colsample_bylevel=None, colsample_bynode=None,\n",
       "              colsample_bytree=None, early_stopping_rounds=None,\n",
       "              enable_categorical=False, eval_metric=&#x27;auc&#x27;, feature_types=None,\n",
       "              gamma=None, gpu_id=None, grow_policy=None, importance_type=None,\n",
       "              interaction_constraints=None, learning_rate=0.3, max_bin=10,\n",
       "              max_cat_threshold=None, max_cat_to_onehot=None,\n",
       "              max_delta_step=None, max_depth=5, max_leaves=None,\n",
       "              min_child_weight=0, missing=nan, monotone_constraints=None,\n",
       "              n_estimators=31, n_jobs=None, num_parallel_tree=None,\n",
       "              predictor=None, random_state=None, ...)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "XGBClassifier(base_score=0.5, booster=None, callbacks=None,\n",
       "              colsample_bylevel=None, colsample_bynode=None,\n",
       "              colsample_bytree=None, early_stopping_rounds=None,\n",
       "              enable_categorical=False, eval_metric='auc', feature_types=None,\n",
       "              gamma=None, gpu_id=None, grow_policy=None, importance_type=None,\n",
       "              interaction_constraints=None, learning_rate=0.3, max_bin=10,\n",
       "              max_cat_threshold=None, max_cat_to_onehot=None,\n",
       "              max_delta_step=None, max_depth=5, max_leaves=None,\n",
       "              min_child_weight=0, missing=nan, monotone_constraints=None,\n",
       "              n_estimators=31, n_jobs=None, num_parallel_tree=None,\n",
       "              predictor=None, random_state=None, ...)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# train on the whole dataset and measure performance\n",
    "\n",
    "# Use \"hist\" for constructing the trees, with early stopping enabled.\n",
    "clf2 = xgb.XGBClassifier(\n",
    "    tree_method=\"hist\",\n",
    "    n_estimators=clf.best_iteration + 1,\n",
    "    max_depth=5,\n",
    "    learning_rate=0.3,\n",
    "    max_bin=10,\n",
    "    eval_metric=\"auc\",\n",
    "    base_score=0.5,\n",
    "    reg_lambda=0.1,\n",
    "    min_child_weight=0,\n",
    ")\n",
    "clf2.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train set AUC score:  0.9361651609369599 num_trees:  31\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    \"train set AUC score: \",\n",
    "    roc_auc_score(y, clf2.predict_proba(X)[:, 1]),\n",
    "    \"num_trees: \",\n",
    "    clf2.best_iteration + 1,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_fed, X_test_fed = train_test_split_fed(data, test_size=0.2, random_state=94)\n",
    "y_train_fed, y_test_fed = train_test_split_fed(label, test_size=0.2, random_state=94)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Try to do the same using sgb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:root:Create proxy actor <class 'secretflow.ml.boost.sgb_v.factory.sgb_actor.SGBActor'> with party alice.\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 0 time 5.8822256390121765s\n",
      "INFO:jax._src.xla_bridge:Unable to initialize backend 'cuda': \n",
      "INFO:jax._src.xla_bridge:Unable to initialize backend 'rocm': module 'jaxlib.xla_extension' has no attribute 'GpuAllocatorConfig'\n",
      "INFO:jax._src.xla_bridge:Unable to initialize backend 'tpu': INTERNAL: Failed to open libtpu.so: libtpu.so: cannot open shared object file: No such file or directory\n",
      "WARNING:jax._src.xla_bridge:An NVIDIA GPU may be present on this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu.\n",
      "INFO:root:[0]\ttrain-roc_auc:0.86629\tval-roc_auc:0.87240\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 1 time 0.22433374699903652s\n",
      "INFO:root:[1]\ttrain-roc_auc:0.88394\tval-roc_auc:0.88925\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 2 time 0.23023367498535663s\n",
      "INFO:root:[2]\ttrain-roc_auc:0.89237\tval-roc_auc:0.89616\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 3 time 4.779345843999181s\n",
      "INFO:root:[3]\ttrain-roc_auc:0.89762\tval-roc_auc:0.89850\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 4 time 0.21870624303119257s\n",
      "INFO:root:[4]\ttrain-roc_auc:0.90027\tval-roc_auc:0.89900\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 5 time 0.2107155859703198s\n",
      "INFO:root:[5]\ttrain-roc_auc:0.90253\tval-roc_auc:0.90005\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 6 time 0.20649968198267743s\n",
      "INFO:root:[6]\ttrain-roc_auc:0.90466\tval-roc_auc:0.90063\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 7 time 0.2591691130073741s\n",
      "INFO:root:[7]\ttrain-roc_auc:0.90649\tval-roc_auc:0.90174\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 8 time 0.23821753094671294s\n",
      "INFO:root:[8]\ttrain-roc_auc:0.91040\tval-roc_auc:0.90510\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 9 time 0.21874118997948244s\n",
      "INFO:root:[9]\ttrain-roc_auc:0.91196\tval-roc_auc:0.90527\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 10 time 0.22477014001924545s\n",
      "INFO:root:[10]\ttrain-roc_auc:0.91338\tval-roc_auc:0.90564\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 11 time 0.2105429110233672s\n",
      "INFO:root:[11]\ttrain-roc_auc:0.91481\tval-roc_auc:0.90548\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 12 time 0.19651963497744873s\n",
      "INFO:root:[12]\ttrain-roc_auc:0.91629\tval-roc_auc:0.90616\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 13 time 0.23939903901191428s\n",
      "INFO:root:[13]\ttrain-roc_auc:0.91797\tval-roc_auc:0.90659\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 14 time 0.19962828402640298s\n",
      "INFO:root:[14]\ttrain-roc_auc:0.91968\tval-roc_auc:0.90802\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 15 time 0.21050006203586236s\n",
      "INFO:root:[15]\ttrain-roc_auc:0.92157\tval-roc_auc:0.90991\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 16 time 0.2094137740205042s\n",
      "INFO:root:[16]\ttrain-roc_auc:0.92286\tval-roc_auc:0.91044\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 17 time 0.21073465299559757s\n",
      "INFO:root:[17]\ttrain-roc_auc:0.92349\tval-roc_auc:0.91042\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 18 time 0.20418725500348955s\n",
      "INFO:root:[18]\ttrain-roc_auc:0.92548\tval-roc_auc:0.91200\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 19 time 0.19339005899382755s\n",
      "INFO:root:[19]\ttrain-roc_auc:0.92764\tval-roc_auc:0.91303\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 20 time 0.19981653901049867s\n",
      "INFO:root:[20]\ttrain-roc_auc:0.92825\tval-roc_auc:0.91301\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 21 time 0.2003304980462417s\n",
      "INFO:root:[21]\ttrain-roc_auc:0.92970\tval-roc_auc:0.91382\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 22 time 0.19011378602590412s\n",
      "INFO:root:[22]\ttrain-roc_auc:0.93086\tval-roc_auc:0.91419\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 23 time 0.23198215302545577s\n",
      "INFO:root:[23]\ttrain-roc_auc:0.93289\tval-roc_auc:0.91602\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 24 time 0.22665166796650738s\n",
      "INFO:root:[24]\ttrain-roc_auc:0.93389\tval-roc_auc:0.91655\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 25 time 0.28643811104120687s\n",
      "INFO:root:[25]\ttrain-roc_auc:0.93459\tval-roc_auc:0.91691\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 26 time 0.1926098190015182s\n",
      "INFO:root:[26]\ttrain-roc_auc:0.93876\tval-roc_auc:0.91942\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 27 time 0.25037117599276826s\n",
      "INFO:root:[27]\ttrain-roc_auc:0.93978\tval-roc_auc:0.91988\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 28 time 0.2111208220012486s\n",
      "INFO:root:[28]\ttrain-roc_auc:0.94045\tval-roc_auc:0.91986\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 29 time 0.20117839600425214s\n",
      "INFO:root:[29]\ttrain-roc_auc:0.94076\tval-roc_auc:0.91979\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 30 time 0.24104506097501144s\n",
      "INFO:root:[30]\ttrain-roc_auc:0.94198\tval-roc_auc:0.92073\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 31 time 0.22020784503547475s\n",
      "INFO:root:[31]\ttrain-roc_auc:0.94270\tval-roc_auc:0.92104\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 32 time 0.22311462298966944s\n",
      "INFO:root:[32]\ttrain-roc_auc:0.94309\tval-roc_auc:0.92115\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 33 time 0.19654598599299788s\n",
      "INFO:root:[33]\ttrain-roc_auc:0.94333\tval-roc_auc:0.92097\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 34 time 0.2035467600217089s\n",
      "INFO:root:[34]\ttrain-roc_auc:0.94385\tval-roc_auc:0.92099\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 35 time 0.21578870801022276s\n",
      "INFO:root:[35]\ttrain-roc_auc:0.94448\tval-roc_auc:0.92082\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 36 time 0.1958170040161349s\n",
      "INFO:root:[36]\ttrain-roc_auc:0.94491\tval-roc_auc:0.92072\n",
      "\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 37 time 0.28233458899194375s\n",
      "INFO:root:early_stopped, current tree num: 38\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36m(HEUSkKeeper(heu_id=124761017292096, party=alice) pid=1777406)\u001b[0m [2024-04-22 15:36:54.817] [info] [thread_pool.cc:30] Create a fixed thread pool with size 63\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[36m(_run pid=1774211)\u001b[0m INFO:jax._src.xla_bridge:Unable to initialize backend 'cuda': \n",
      "\u001b[36m(_run pid=1774211)\u001b[0m INFO:jax._src.xla_bridge:Unable to initialize backend 'rocm': module 'jaxlib.xla_extension' has no attribute 'GpuAllocatorConfig'\n",
      "\u001b[36m(_run pid=1774211)\u001b[0m INFO:jax._src.xla_bridge:Unable to initialize backend 'tpu': INTERNAL: Failed to open libtpu.so: libtpu.so: cannot open shared object file: No such file or directory\n",
      "\u001b[36m(_run pid=1774211)\u001b[0m WARNING:jax._src.xla_bridge:An NVIDIA GPU may be present on this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu.\n"
     ]
    }
   ],
   "source": [
    "sgb = Sgb(heu)\n",
    "params = get_classic_XGB_params()\n",
    "params['num_boost_round'] = 100\n",
    "params['max_depth'] = 5\n",
    "params['base_score'] = 0.5\n",
    "params['reg_lambda'] = 0.1\n",
    "params['learning_rate'] = 0.3\n",
    "params['sketch_eps'] = 1 / 10\n",
    "params['enable_early_stop'] = True\n",
    "params['enable_monitor'] = True\n",
    "params['validation_fraction'] = 0.2\n",
    "params['stopping_rounds'] = 5\n",
    "params['stopping_tolerance'] = 0.0000000001\n",
    "params['seed'] = 94\n",
    "params['first_tree_with_label_holder_feature'] = False\n",
    "params['save_best_model'] = True\n",
    "\n",
    "model = sgb.train(params, X_train_fed, y_train_fed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train set AUC score:  0.9381787372071879 test set AUC score:  0.9198223624599511 num_trees:  33\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    \"train set AUC score: \",\n",
    "    roc_auc_score(\n",
    "        reveal(y_train_fed.partitions[alice].data), reveal(model.predict(X_train_fed))\n",
    "    ),\n",
    "    \"test set AUC score: \",\n",
    "    roc_auc_score(\n",
    "        reveal(y_test_fed.partitions[alice].data), reveal(model.predict(X_test_fed))\n",
    "    ),\n",
    "    \"num_trees: \",\n",
    "    len(model.get_trees()),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:root:Create proxy actor <class 'secretflow.ml.boost.sgb_v.factory.sgb_actor.SGBActor'> with party alice.\n",
      "INFO:root:train tree context set up.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 0 time 1.5635340110165998s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 1 time 0.33637010597158223s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 2 time 0.3417563459952362s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 3 time 0.3387716009747237s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 4 time 0.34694120701169595s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 5 time 0.34902988199610263s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 6 time 0.3459926190553233s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 7 time 0.33632252702955157s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 8 time 0.32081323000602424s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 9 time 0.34241603896953166s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 10 time 0.3063079469720833s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 11 time 0.3138028900139034s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 12 time 0.31306355696870014s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 13 time 0.3181340919691138s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 14 time 0.31928949896246195s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 15 time 0.30834720900747925s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 16 time 0.31783646100666374s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 17 time 0.2962478280533105s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 18 time 0.30946173903066665s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 19 time 0.30028603901155293s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 20 time 0.2980068330070935s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 21 time 0.3230232660425827s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 22 time 0.3061675620265305s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 23 time 0.30418004596140236s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 24 time 0.29429697297746316s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 25 time 0.2893914590240456s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 26 time 0.3032367390114814s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 27 time 0.3122948190430179s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 28 time 0.31372399197425693s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 29 time 0.3103683390072547s\n",
      "INFO:root:train tree context set up.\n",
      "INFO:root:begin train tree.\n",
      "INFO:root:epoch 30 time 0.28349055896978825s\n"
     ]
    }
   ],
   "source": [
    "sgb = Sgb(heu)\n",
    "params = get_classic_XGB_params()\n",
    "params['num_boost_round'] = clf.best_iteration + 1\n",
    "params['max_depth'] = 5\n",
    "params['base_score'] = 0.5\n",
    "params['learning_rate'] = 0.3\n",
    "params['sketch_eps'] = 1 / 10\n",
    "params['seed'] = 94\n",
    "params['first_tree_with_label_holder_feature'] = False\n",
    "\n",
    "model2 = sgb.train(params, data, label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train set AUC score:  0.9342629405465066 num_trees:  31\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    \"train set AUC score: \",\n",
    "    roc_auc_score(reveal(label.partitions[alice].data), reveal(model2.predict(data))),\n",
    "    \"num_trees: \",\n",
    "    len(model2.get_trees()),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pipeline the test run\n",
    "\n",
    "We have performed one test run on the bank dataset. In this test, we use the same parameters for XGB and SGB and compare the model results.\n",
    "There are several steps:\n",
    "\n",
    "1. run XGB on a set of params on training data with early stopping enabled\n",
    "2. run SGB on a set of params on training data with early stopping enabled\n",
    "3. run XGB on the whole training data (no early stopping) with the optimal rounds from step 1\n",
    "4. run SGB on the whole training data (no early stopping) with the optimal rounds from step 1\n",
    "6. collect results of convergence rounds and AUC scores\n",
    "\n",
    "We have found the convergence rounds are close and AUC scores are similar between XGB and SGB, therefore add the evidence SGB is similar to XGB in terms of accuracy.\n",
    "\n",
    "However, a single data point is not enough to make a conclusion about the performance of SGB.\n",
    "It is possible to perform multiple runs and collect the data.\n",
    "\n",
    "Now we are going to pipeline the test run process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we begin with a single np dataset\n",
    "# we use label encoding for all the categorical variables\n",
    "# we separate the labels from the features\n",
    "# we then give it to alice and bob by halves, in particular give alice the label\n",
    "\n",
    "\n",
    "def preprocess_data(dataset: pd.DataFrame, label_column: str):\n",
    "    # perform label encoding on categorical variables\n",
    "    for col in dataset.columns:\n",
    "        if isinstance(dataset[col][0], str):\n",
    "            le = SkLabelEncoder()\n",
    "            dataset[col] = le.fit_transform(dataset[col])\n",
    "\n",
    "    # separate labels from features\n",
    "    X = dataset.drop(label_column, axis=1)\n",
    "    y = dataset[label_column]\n",
    "\n",
    "    X_col_num = X.shape[1]\n",
    "    split_count = int(X_col_num / 2)\n",
    "    vdata = VDataFrame(\n",
    "        partitions={\n",
    "            alice: partition(alice(lambda x: x)(X.iloc[:, :split_count])),\n",
    "            bob: partition(bob(lambda x: x)(X.iloc[:, split_count:])),\n",
    "        }\n",
    "    )\n",
    "    label = VDataFrame(partitions={alice: partition(alice(lambda x: x)(y))})\n",
    "\n",
    "    return X, y, vdata, label\n",
    "\n",
    "\n",
    "DEFAULT_XGB_PARAMS = {\n",
    "    \"tree_method\": \"hist\",\n",
    "    \"n_estimators\": 100,\n",
    "    \"max_depth\": 5,\n",
    "    \"learning_rate\": 0.3,\n",
    "    \"max_bin\": 10,\n",
    "    \"early_stopping_rounds\": 5,\n",
    "    \"base_score\": 0.5,\n",
    "    \"eval_metric\": \"auc\",\n",
    "    \"reg_lambda\": 0.1,\n",
    "    \"min_child_weight\": 0,\n",
    "    \"random_state\": 95,\n",
    "}\n",
    "\n",
    "\n",
    "def fit_xgb(X, y, params, valid_frac=0.2, test_frac=0.2):\n",
    "    X_train, X_test, y_train, y_test = train_test_split(\n",
    "        X, y, test_size=test_frac, random_state=params[\"random_state\"]\n",
    "    )\n",
    "    X_train, X_val, y_train, y_val = train_test_split(\n",
    "        X_train, y_train, test_size=valid_frac, random_state=params[\"random_state\"]\n",
    "    )\n",
    "\n",
    "    # Use \"hist\" for constructing the trees, with early stopping enabled.\n",
    "    clf = xgb.XGBClassifier(**params)\n",
    "    clf.fit(X_train, y_train, eval_set=[(X_val, y_val)])\n",
    "    return clf, X_test, y_test\n",
    "\n",
    "\n",
    "def find_xgb_auc_and_best_iter_round(clf, X_test, y_test):\n",
    "    converge_num_trees = clf.best_iteration + 1\n",
    "    converge_test_auc = roc_auc_score(y_test, clf.predict_proba(X_test)[:, 1])\n",
    "    return converge_test_auc, converge_num_trees\n",
    "\n",
    "\n",
    "def fit_sgb(vdata, label, params, valid_frac=0.2, test_frac=0.2):\n",
    "    X_train_fed, X_test_fed = train_test_split_fed(\n",
    "        vdata, test_size=test_frac, random_state=params[\"random_state\"]\n",
    "    )\n",
    "    y_train_fed, y_test_fed = train_test_split_fed(\n",
    "        label, test_size=test_frac, random_state=params[\"random_state\"]\n",
    "    )\n",
    "    sgb = Sgb(heu)\n",
    "    sgb_params = xgb_params_converter(params)\n",
    "    sgb_params['validation_fraction'] = valid_frac\n",
    "    model = sgb.train(sgb_params, X_train_fed, y_train_fed)\n",
    "    return model, X_test_fed, y_test_fed\n",
    "\n",
    "\n",
    "def find_sgb_auc_and_best_iter_round(model, X_test_fed, y_test_fed):\n",
    "    test_auc = roc_auc_score(\n",
    "        reveal(y_test_fed.partitions[alice].data), reveal(model.predict(X_test_fed))\n",
    "    )\n",
    "    num_trees = len(model.get_trees())\n",
    "    return test_auc, num_trees\n",
    "\n",
    "\n",
    "def find_sgb_auc_at_xgb_convergence_point(\n",
    "    model, X_test_fed, y_test_fed, xgb_converge_num_trees\n",
    "):\n",
    "    return roc_auc_score(\n",
    "        reveal(y_test_fed.partitions[alice].data),\n",
    "        reveal(model[:xgb_converge_num_trees].predict(X_test_fed)),\n",
    "    )\n",
    "\n",
    "\n",
    "class ExperimentResult:\n",
    "    def __init__(self):\n",
    "        self.xgb_test_auc = 0\n",
    "        self.xgb_num_trees = 0\n",
    "        self.xgboost_fit_time = 0\n",
    "\n",
    "        self.sgb_test_auc = 0\n",
    "        self.sgb_num_trees = 0\n",
    "        self.sgb_fit_time = 0\n",
    "        self.sgb_test_auc_at_xgb_convergence = 0\n",
    "\n",
    "    def to_dict(self):\n",
    "        return {\n",
    "            'xgb_test_auc': self.xgb_test_auc,\n",
    "            'xgb_num_trees': self.xgb_num_trees,\n",
    "            'xgboost_fit_time': self.xgboost_fit_time,\n",
    "            'sgb_test_auc': self.sgb_test_auc,\n",
    "            'sgb_num_trees': self.sgb_num_trees,\n",
    "            'sgb_fit_time': self.sgb_fit_time,\n",
    "            'sgb_test_auc_at_xgb_convergence': self.sgb_test_auc_at_xgb_convergence,\n",
    "        }\n",
    "\n",
    "\n",
    "def run_experiement(\n",
    "    dataset: pd.DataFrame,\n",
    "    label_column: str,\n",
    "    params: dict,\n",
    "    experiment_name: str,\n",
    "    valid_frac=0.2,\n",
    "    test_frac=0.2,\n",
    "):\n",
    "    X, y, vdata, label = preprocess_data(dataset, label_column)\n",
    "    print(\"Starting {}\".format(experiment_name))\n",
    "\n",
    "    start = time.perf_counter()\n",
    "    clf, X_test, y_test = fit_xgb(X, y, params, valid_frac, test_frac)\n",
    "    xgboost_fit_time = time.perf_counter() - start\n",
    "    xgb_test_auc, xgb_num_trees = find_xgb_auc_and_best_iter_round(clf, X_test, y_test)\n",
    "    print(\"XGBoost Test AUC: {}, Num Trees: {}\".format(xgb_test_auc, xgb_num_trees))\n",
    "\n",
    "    start = time.perf_counter()\n",
    "    model, X_test_fed, y_test_fed = fit_sgb(vdata, label, params, valid_frac, test_frac)\n",
    "    sgb_fit_time = time.perf_counter() - start\n",
    "    sgb_test_auc, sgb_num_trees = find_sgb_auc_and_best_iter_round(\n",
    "        model, X_test_fed, y_test_fed\n",
    "    )\n",
    "    print(\"SGB Test AUC: {}, Num Trees: {}\".format(sgb_test_auc, sgb_num_trees))\n",
    "\n",
    "    sgb_test_auc_at_xgb_convergence = find_sgb_auc_at_xgb_convergence_point(\n",
    "        model, X_test_fed, y_test_fed, xgb_num_trees\n",
    "    )\n",
    "    print(\n",
    "        \"SGB Test AUC at XGB Convergence Point: {}\".format(\n",
    "            sgb_test_auc_at_xgb_convergence\n",
    "        )\n",
    "    )\n",
    "\n",
    "    experiment_result = ExperimentResult()\n",
    "\n",
    "    experiment_result.xgb_test_auc = xgb_test_auc\n",
    "    experiment_result.xgb_num_trees = xgb_num_trees\n",
    "    experiment_result.xgboost_fit_time = xgboost_fit_time\n",
    "\n",
    "    experiment_result.sgb_test_auc = sgb_test_auc\n",
    "    experiment_result.sgb_num_trees = sgb_num_trees\n",
    "    experiment_result.sgb_fit_time = sgb_fit_time\n",
    "\n",
    "    experiment_result.sgb_test_auc_at_xgb_convergence = sgb_test_auc_at_xgb_convergence\n",
    "    return experiment_result\n",
    "\n",
    "\n",
    "def collect_results(results: List[ExperimentResult]):\n",
    "    results_dict = [r.to_dict() for r in results]\n",
    "    return pd.DataFrame(results_dict)\n",
    "\n",
    "\n",
    "def run_repeated_experiments(\n",
    "    dataset: pd.DataFrame,\n",
    "    label_column: str,\n",
    "    params: dict,\n",
    "    experiment_name: str,\n",
    "    num_repeats: int,\n",
    "    valid_frac=0.2,\n",
    "    test_frac=0.2,\n",
    "    seed=1212,\n",
    "):\n",
    "    np.random.seed(seed)\n",
    "\n",
    "    results = []\n",
    "    for i in range(num_repeats):\n",
    "        random_state = np.random.randint(low=0, high=1000000)\n",
    "        params[\"random_state\"] = random_state\n",
    "        result = run_experiement(\n",
    "            dataset,\n",
    "            label_column,\n",
    "            params,\n",
    "            experiment_name + \"_repeat_\" + str(i + 1),\n",
    "            valid_frac,\n",
    "            test_frac,\n",
    "        )\n",
    "        results.append(result)\n",
    "    return collect_results(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# outputs are cleared in web demo for viewing purposes\n",
    "\n",
    "creditcard_experiement_results_table = run_repeated_experiments(\n",
    "    load_creditcard_unpartitioned(), 'Class', DEFAULT_XGB_PARAMS, 'creditcard', 20\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# outputs are cleared in web demo for viewing purposes\n",
    "\n",
    "bank_marketing_experiement_results_table = run_repeated_experiments(\n",
    "    load_bank_marketing_unpartitioned(full=True),\n",
    "    'y',\n",
    "    DEFAULT_XGB_PARAMS,\n",
    "    'bank_marketing',\n",
    "    15,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>xgb_test_auc</th>\n",
       "      <th>xgb_num_trees</th>\n",
       "      <th>xgboost_fit_time</th>\n",
       "      <th>sgb_test_auc</th>\n",
       "      <th>sgb_num_trees</th>\n",
       "      <th>sgb_fit_time</th>\n",
       "      <th>sgb_test_auc_at_xgb_convergence</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>20.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>20.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.962845</td>\n",
       "      <td>8.050000</td>\n",
       "      <td>2.212415</td>\n",
       "      <td>0.963318</td>\n",
       "      <td>8.400000</td>\n",
       "      <td>38.845686</td>\n",
       "      <td>0.961961</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.014806</td>\n",
       "      <td>6.629083</td>\n",
       "      <td>1.956283</td>\n",
       "      <td>0.012678</td>\n",
       "      <td>7.014271</td>\n",
       "      <td>21.883654</td>\n",
       "      <td>0.012365</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.931132</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.200580</td>\n",
       "      <td>0.939873</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>18.894895</td>\n",
       "      <td>0.939873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.952101</td>\n",
       "      <td>3.750000</td>\n",
       "      <td>1.273729</td>\n",
       "      <td>0.957149</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>25.896972</td>\n",
       "      <td>0.954517</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.964033</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>1.366425</td>\n",
       "      <td>0.964311</td>\n",
       "      <td>6.500000</td>\n",
       "      <td>33.616968</td>\n",
       "      <td>0.962278</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.974693</td>\n",
       "      <td>10.250000</td>\n",
       "      <td>1.887061</td>\n",
       "      <td>0.972549</td>\n",
       "      <td>9.250000</td>\n",
       "      <td>39.687874</td>\n",
       "      <td>0.968840</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.989315</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>9.191370</td>\n",
       "      <td>0.984785</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>114.188385</td>\n",
       "      <td>0.984785</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       xgb_test_auc  xgb_num_trees  xgboost_fit_time  sgb_test_auc  \\\n",
       "count     20.000000      20.000000         20.000000     20.000000   \n",
       "mean       0.962845       8.050000          2.212415      0.963318   \n",
       "std        0.014806       6.629083          1.956283      0.012678   \n",
       "min        0.931132       1.000000          1.200580      0.939873   \n",
       "25%        0.952101       3.750000          1.273729      0.957149   \n",
       "50%        0.964033       6.000000          1.366425      0.964311   \n",
       "75%        0.974693      10.250000          1.887061      0.972549   \n",
       "max        0.989315      26.000000          9.191370      0.984785   \n",
       "\n",
       "       sgb_num_trees  sgb_fit_time  sgb_test_auc_at_xgb_convergence  \n",
       "count      20.000000     20.000000                        20.000000  \n",
       "mean        8.400000     38.845686                         0.961961  \n",
       "std         7.014271     21.883654                         0.012365  \n",
       "min         2.000000     18.894895                         0.939873  \n",
       "25%         4.000000     25.896972                         0.954517  \n",
       "50%         6.500000     33.616968                         0.962278  \n",
       "75%         9.250000     39.687874                         0.968840  \n",
       "max        32.000000    114.188385                         0.984785  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "creditcard_experiement_results_table.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>xgb_test_auc</th>\n",
       "      <th>xgb_num_trees</th>\n",
       "      <th>xgboost_fit_time</th>\n",
       "      <th>sgb_test_auc</th>\n",
       "      <th>sgb_num_trees</th>\n",
       "      <th>sgb_fit_time</th>\n",
       "      <th>sgb_test_auc_at_xgb_convergence</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>15.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>15.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.923508</td>\n",
       "      <td>43.533333</td>\n",
       "      <td>3.831747</td>\n",
       "      <td>0.919628</td>\n",
       "      <td>45.133333</td>\n",
       "      <td>87.564592</td>\n",
       "      <td>0.918430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.002130</td>\n",
       "      <td>9.620415</td>\n",
       "      <td>5.015576</td>\n",
       "      <td>0.002174</td>\n",
       "      <td>8.983053</td>\n",
       "      <td>13.680772</td>\n",
       "      <td>0.002543</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.920564</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>1.345067</td>\n",
       "      <td>0.914188</td>\n",
       "      <td>34.000000</td>\n",
       "      <td>71.390456</td>\n",
       "      <td>0.914188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.922071</td>\n",
       "      <td>38.500000</td>\n",
       "      <td>1.389431</td>\n",
       "      <td>0.918462</td>\n",
       "      <td>37.000000</td>\n",
       "      <td>75.477611</td>\n",
       "      <td>0.916609</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.922776</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>1.472737</td>\n",
       "      <td>0.919908</td>\n",
       "      <td>44.000000</td>\n",
       "      <td>88.222559</td>\n",
       "      <td>0.918018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.925062</td>\n",
       "      <td>47.500000</td>\n",
       "      <td>1.599777</td>\n",
       "      <td>0.921103</td>\n",
       "      <td>49.500000</td>\n",
       "      <td>95.262573</td>\n",
       "      <td>0.920205</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.927874</td>\n",
       "      <td>65.000000</td>\n",
       "      <td>15.394586</td>\n",
       "      <td>0.922562</td>\n",
       "      <td>62.000000</td>\n",
       "      <td>113.003852</td>\n",
       "      <td>0.922562</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       xgb_test_auc  xgb_num_trees  xgboost_fit_time  sgb_test_auc  \\\n",
       "count     15.000000      15.000000         15.000000     15.000000   \n",
       "mean       0.923508      43.533333          3.831747      0.919628   \n",
       "std        0.002130       9.620415          5.015576      0.002174   \n",
       "min        0.920564      28.000000          1.345067      0.914188   \n",
       "25%        0.922071      38.500000          1.389431      0.918462   \n",
       "50%        0.922776      43.000000          1.472737      0.919908   \n",
       "75%        0.925062      47.500000          1.599777      0.921103   \n",
       "max        0.927874      65.000000         15.394586      0.922562   \n",
       "\n",
       "       sgb_num_trees  sgb_fit_time  sgb_test_auc_at_xgb_convergence  \n",
       "count      15.000000     15.000000                        15.000000  \n",
       "mean       45.133333     87.564592                         0.918430  \n",
       "std         8.983053     13.680772                         0.002543  \n",
       "min        34.000000     71.390456                         0.914188  \n",
       "25%        37.000000     75.477611                         0.916609  \n",
       "50%        44.000000     88.222559                         0.918018  \n",
       "75%        49.500000     95.262573                         0.920205  \n",
       "max        62.000000    113.003852                         0.922562  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bank_marketing_experiement_results_table.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "\n",
    "def xy_to_dataframe(X, y, feature_names=None):\n",
    "    \"\"\"\n",
    "    Convert (X, y) into a pandas DataFrame.\n",
    "\n",
    "    Parameters:\n",
    "    - X (array-like): A two-dimensional array-like structure containing feature data.\n",
    "    - y (array-like): A one-dimensional array-like structure containing target variable.\n",
    "    - feature_names (list of str, optional): A list of feature names for the DataFrame columns. If None, generic names are used.\n",
    "\n",
    "    Returns:\n",
    "    - df (pd.DataFrame): A pandas DataFrame containing X and y combined.\n",
    "    \"\"\"\n",
    "\n",
    "    # Check if feature_names is provided; if not, create generic feature names\n",
    "    if feature_names is None:\n",
    "        feature_names = [f\"feature_{i}\" for i in range(1, len(X[0]) + 1)]\n",
    "    elif len(feature_names) != len(X[0]):\n",
    "        raise ValueError(\n",
    "            \"Length of feature_names does not match number of features in X.\"\n",
    "        )\n",
    "\n",
    "    # Convert X and y into a pandas DataFrame\n",
    "    df_X = pd.DataFrame(X, columns=feature_names)\n",
    "    df_y = pd.Series(y, name='target')\n",
    "\n",
    "    # Concatenate X and y dataframes\n",
    "    df = pd.concat([df_X, df_y], axis=1)\n",
    "\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# outputs are cleared for viewing purpose\n",
    "\n",
    "from sklearn.datasets import make_classification\n",
    "\n",
    "random_200w_200d_experiement_results_table = run_repeated_experiments(\n",
    "    xy_to_dataframe(\n",
    "        *make_classification(n_samples=200 * 10000, n_features=200, random_state=42)\n",
    "    ),\n",
    "    'target',\n",
    "    DEFAULT_XGB_PARAMS,\n",
    "    'random_200w_200d',\n",
    "    10,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>xgb_test_auc</th>\n",
       "      <th>xgb_num_trees</th>\n",
       "      <th>xgboost_fit_time</th>\n",
       "      <th>sgb_test_auc</th>\n",
       "      <th>sgb_num_trees</th>\n",
       "      <th>sgb_fit_time</th>\n",
       "      <th>sgb_test_auc_at_xgb_convergence</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>10.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>10.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.957205</td>\n",
       "      <td>21.900000</td>\n",
       "      <td>10.489245</td>\n",
       "      <td>0.957177</td>\n",
       "      <td>22.600000</td>\n",
       "      <td>755.863760</td>\n",
       "      <td>0.957171</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.000227</td>\n",
       "      <td>1.197219</td>\n",
       "      <td>3.330223</td>\n",
       "      <td>0.000194</td>\n",
       "      <td>1.429841</td>\n",
       "      <td>46.846637</td>\n",
       "      <td>0.000197</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.956807</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>7.131368</td>\n",
       "      <td>0.956894</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>707.297945</td>\n",
       "      <td>0.956894</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.957049</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>7.368822</td>\n",
       "      <td>0.957043</td>\n",
       "      <td>21.250000</td>\n",
       "      <td>720.436065</td>\n",
       "      <td>0.957028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.957252</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>10.184636</td>\n",
       "      <td>0.957181</td>\n",
       "      <td>22.500000</td>\n",
       "      <td>746.350113</td>\n",
       "      <td>0.957167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.957362</td>\n",
       "      <td>22.750000</td>\n",
       "      <td>13.473267</td>\n",
       "      <td>0.957306</td>\n",
       "      <td>23.750000</td>\n",
       "      <td>781.101195</td>\n",
       "      <td>0.957309</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.957541</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>14.274549</td>\n",
       "      <td>0.957511</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>857.693256</td>\n",
       "      <td>0.957511</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       xgb_test_auc  xgb_num_trees  xgboost_fit_time  sgb_test_auc  \\\n",
       "count     10.000000      10.000000         10.000000     10.000000   \n",
       "mean       0.957205      21.900000         10.489245      0.957177   \n",
       "std        0.000227       1.197219          3.330223      0.000194   \n",
       "min        0.956807      20.000000          7.131368      0.956894   \n",
       "25%        0.957049      21.000000          7.368822      0.957043   \n",
       "50%        0.957252      22.000000         10.184636      0.957181   \n",
       "75%        0.957362      22.750000         13.473267      0.957306   \n",
       "max        0.957541      24.000000         14.274549      0.957511   \n",
       "\n",
       "       sgb_num_trees  sgb_fit_time  sgb_test_auc_at_xgb_convergence  \n",
       "count      10.000000     10.000000                        10.000000  \n",
       "mean       22.600000    755.863760                         0.957171  \n",
       "std         1.429841     46.846637                         0.000197  \n",
       "min        21.000000    707.297945                         0.956894  \n",
       "25%        21.250000    720.436065                         0.957028  \n",
       "50%        22.500000    746.350113                         0.957167  \n",
       "75%        23.750000    781.101195                         0.957309  \n",
       "max        25.000000    857.693256                         0.957511  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random_200w_200d_experiement_results_table.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "As we can see, the sgb can perform similarly to XGBoost models. \n",
    "\n",
    "However, it is not as fast as XGBoost. The ratio between the time consumptions can range from 8 to 12 times in a LAN setting.\n",
    "\n",
    "Welcome to contribute and run more analysis on more datasets and parameters!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py38",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
